Joint Attention in Human-Robot Interaction

We propose a computational model of joint attention consisting of three parts: responding to joint attention, initiating joint attention, and ensuring joint attention. This model is supported by psychological findings and matches the developmental timeline in humans. We present two experiments that test this model and investigate joint attention in human-robot interaction. The first experiment explored the effects of responding to joint attention on human-robot interaction. We show that robots responding to joint attention are more transparent to humans and are more competent and socially interactive. The second experiment studied the importance of ensuring joint attention in human-robot interaction. Data upheld our hypotheses that a robot’s ensuring joint attention behavior yields better performance in human-robot interactive tasks and ensuring joint attention behaviors are perceived as natural behaviors. Joint attention, a process to share one’s current attention with another by using social cues such as gaze, has been recognized as a crucial component in interactions and an important milestone in infant development. One of the key components of Autism Spectrum Disorder is the failure to develop joint attention capabilities. It is hypothesized that the failure to develop this fundamental social skill leads people on the autism spectrum to often have difficulties in communication and interaction with other people (Baron-Cohen 1997). Therefore, to facilitate natural human-robot interaction (HRI), we believe that robots need social skills to respond to, initiate, and maintain joint attention with humans. Our model of joint attention reflects the complexity of this social skill for cognitive robots, dividing the skill into its three main components: responding to joint attention (RJA), initiating joint attention (IJA), and ensuring joint attention (EJA). RJA is the ability to follow another’s direction of gaze and gestures in order to attain common experience. IJA is the ability to manipulate another’s attention to a focus of interest in order to share experience. EJA is the ability to monitor another’s attention and to ensure that the state of joint attention is reached. These match the developmental milestones of joint attention in infancy. Infants start with the skill of following a care-giver’s gaze, and then they exhibit Copyright c © 2010, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. pointing gestures (especially declarative pointing gestures) to get the care-giver’s attention. Importantly, the initiating actions often come with an ensuring behavior that is to look back and forth between the care-giver and the referential object (Mundy and Newell 2007) We conducted two experiments to test our model and to investigate joint attention in HRI. The first experiment explored the effects of responding to joint attention. Results showed that a robot responding to joint attention is more transparent to people such that interactive task performance is better and people are more confident in the robot’s task success. In addition, people perceive a robot responding to joint attention as more competent and socially interactive. The second experiment studied the importance of ensuring joint attention in HRI. Results suggested that ensuring joint attention yields better task performance and is viewed as a natural behavior in interactions.